Machine Learning Theory CS6783
News :
  1. Welcome to first day of class!
  2. Join ED Discussions Via Canvas

Location and Time :
Location : Hollister Hall, Room 320

Online: The class will also be available via zoom only for Cornell Tech students on request
Time : Tue-Thu 1:25 PM to 2:40 PM (EST)

Office Hours : Wed 2-3pm at 477 CIS Building (new building).

TA: August Chen
Office Hours : TBD

Description : We will discuss both classical results and recent advances in both statistical (iid batch) and online learning theory. The course aims at providing students with tools and techniques to understand inherent mathematical complexities of learning problems, to analyze and prove performance guarantees for machine learning methods and to develop theoretically sound learning algorithms.

Pre-requisite : Student requires a basic level of mathematical maturity and ease/familiarity with theorems and proofs style material. Familiarity with probability theory, basics of algorithms and an introductory course on Machine Learning (CS 4780 or equivalent) are required. M.Eng. and undergraduate students require permission of instructor.

Grading :
Assignments : There will be a total of 4 assignments covering 40% of your grade.

Term project :
    There will be a term project due by the end of the course. The project is worth 52% of your grade. The project could be your choice of research problem approved by me for which you will submit a report by end of term. In the second week of the course I will also give a list of suggestions for broad topics that students can choose from. Groups of at most 3 students per project.
Paper Reading : There will be a total of 2 paper reading covering 8% of your grade (with quizzes for each one).



Lectures/Schedule :
Date Topic Notes Assignments
8/26/25 Introduction, course details, A Bit of Fun [Lec 1]
[Course Info]
Ref: [1] (ch 2) 		Assignment 0 Out
Not for grade, no submnission
8/28/25 Bit Prediction and Cover's Lemma [Lec 2]
Ref: [1] (ch 2)
9/02/25 NO LECTURE NO LECTURE
9/04/25 Cover's Lemma, Rademacher Complexity and Betting Problem [Lec 3]
Ref: [1] (ch 2) 		Assignment 1 Out
Due 16th Sep, 2025
9/09/25
9/11/25
9/16/25 Assignment 1 Due Today
9/18/25
9/23/25 Assignment 2 Out
Due 7th Oct, 2025
9/25/25
9/30/25
10/02/25
10/07/25 Assignment 2 Due Today
10/09/25 Assignment 3 Out
Due 23rd Oct, 2025
10/14/25 FALL BREAK NO LECTURE
10/16/25 Initial Project Idea Proposal Due
10/21/25
10/23/25 Assignment 3 Due today
10/28/25 Assignment 4 Out
Due 11th Nov 2025
10/30/25
11/04/25
11/06/25
11/11/25 Assignment 4 Due today
11/13/25
11/18/25
11/20/25
11/25/25
11/27/25 THANKSGIVING BREAK NO LECTURE
12/02/25
12/04/25


Reference Material :

  1. Statistical Learning and Sequential Prediction, A. Rakhlin and K. Sridharan [pdf]

  2. Introduction to Statistical Learning Theory, O. Bousquet, S. Boucheron, and G. Lugosi [pdf]

  3. Prediction Learning and Games, N. Cesa-Bianchi and G. Lugosi [link]

  4. Understanding Machine Learning From Theory to Algorithms, S. Ben David and S. Shalev-Shwartz [link]

  5. Introduction to Online Convex Optimization, Elad Hazan [link]

  6. Concentration inequalities, S. Boucheron, O. Bousquet, and G. Lugosi [pdf]

  7. A Gentle Introduction to Concentration Inequalities, K. Sridharan [pdf]

  8. On the Vapnik-Chervonenkis-Sauer Lemma by Leon Bottou [link]